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fin  Approximate  Solution  Technique  for  the  Constrained  Search  Path 
Mouing  Target  Search  Problem 

James  H.  Eagle 
James  R.  Vee 

Department  of  Operations  Research 
Naval  Postgraduate  School 
flonterey,  CR  93943 

R  discrete  time  search  is  conducted  for  a  target  mouing  among  a 
finite  set  of  cells  C  =  {1,...,N}.  fit  the  beginning  of  each  time 
period  one  cell  is  searched.  If  cell  i  was  searched  in  the  previous 
time  period,  the  current  search  cell  must  be  selected  from  the  set 
Cj  c.  C,  If  the  target  is  in  the  selected  cell  k,  it  is  detected 
( i . e . ,  found)  with  probabi I  i ty  qj<  €  [0, 1  ] .  I f  the  target  is  not  in 
the  cell  searched,  it  can  not  be  detected  during  the  current  time 
period.  After  an  unsuccessful  search,  a  target  in  cell  i  moves  to 
cell  j  with  probability  Yij  for  the  next  time  period.  The  transition 
probability  matrix  T  =  and  the  initial  distribution  of  the 

target  over  the  search  cells  are  known  to  the  searcher.  The 
objective  of  the  searcher  is  to  select  a  T-tiroe  period  search  path 
which  minimizes  the  probability  of  nondetection, 

1. Background 

The  path  constrained  search  problem,  described  above,  is  a 
difficult  one  to  solve  efficiently.  Trummel  and  Ueisinger  [1985] 
showed  that  the  path  constrained  search  problem  with  a  stationary 
target  is  NP-compiete.  The  moving  target  problem,  which  is  a 
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generalization  of  the  stationary  target  problem,  is  then  also  NP- 
complete. 

Other  than  total  enumeration  of  all  search  paths,  the  only 
optimal  solution  technique  mentioned  in  the  literature  for  the 
mooing  target  constrained  search  problem  has  been  the  dynamic 
programming  procedure  of  Eaglet  1984a] ,  Although  this  method  can 
solve  problems  much  more  quickly  than  total  enumeration,  it  can 
require  a  large  amount  of  computer  storage  as  problem  size 
increases. 

It  was  the  difficulty  experienced  u/ith  soloing  large  problems 
optimally  that  motioated  the  deoelopment  of  good  subopt imal  solution 
procedures.  The  first  such  method  proposed  mas  a  modified  branch- 
and-bound  method  by  Stewart  [1979].  Stewart  used  a  discrete  oersion 
of  a  mooing  target  search  algorithm  gioen  by  Broom  [I960]  to  prooide 
bounds  for  his  procedure.  However,  Broum's  algorithm  does  not 
necessarily  gioe  optimal  solutions  when  search  effort  is  discrete, 
so  these  "bounds"  may  result  in  an  optimal  branch  of  the  enumeration 
tree  being  mistakenly  fathomed,  Nonetheless,  Stewart's 
computational  experience  with  1 -dimensional  search  problems 
indicates  that  the  method  can  perform  well. 

Another  approximate  procedure  was  gioen  by  Eagle  [1984b].  This 
dynamic  programming  method  uses  a  moving  or  "rolling"  time  horizon 
that  greatly  reduces  the  computer  storage  requirements.  It  was  used 
to  approximately  solve  a  small  2-dimensional  problem  (3  by  3  search 
grid)  for  40  time  periods.  This  procedure  generalizes  myopic  search 
by  selecting  in  each  time  period  the  next  cell  to  be  searched  under 
the  assumption  that  the  search  ends  m  time  periods  in  the  future. 
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For  myopic  search,  m  is  1.  For  small  enough  m,  this  procedure  can 
be  implemented  on  a  microcomputer . 

Reported  here  is  a  third  subopt imal  solution  method  which,  like 
Brown's  algorithm,  is  deriued  from  a  nonlinear  programming 
formulation  of  the  search  problem.  Unlike  the  problem  addressed  by 
Brown,  however,  this  formulation  (a)  allows  for  path  constraints  to 
be  specified  for  the  searcher,  (b)  does  not  allow  search  effort  in 
each  time  period  to  be  infinitely  divisible  over  the  search  cells, 
and  (c)  does  not  have  a  convex  detection  function.  Consequently 
the  objective  function  of  the  nonlinear  program  is  not  necessarily 
convex  and  the  solutions  obtained  may  be  local  rather  than  global 
optima.  But  like  Brown's  method,  the  structure  of  the  problem 
allows  a  simple  implementation  of  the  nonlinear  programming  solution 
technique.  Uhen  considered  without  path  constraints,  this  procedure 
has  similarities  to  discrete  versions  of  both  Brown's  algorithm  and 
those  of  Uashburn  [1980]  and  [1983]. 

2.  Definitions 

The  movement  of  the  searcher  is  described  by  a  nonhomogeneous 
harkov  process.  Let  $jj(t)  be  the  probability  that  the  searcher  will 

search  cell  j  in  the  time  period  t,  given  that  cell  i  was  searched  in 
time  period  t-J.  Then  a  search  plan.  S  *  {S(1 ), . , . ,S(T)},  is  a 
sequence  of  T  NxN  stochastic  matrices  satisfying 


A 


is  a  S  composed  entirely  of  ones 


and  zeros. 

Pij(t)  is  the  joint  probability  that,  after  the  search  and 
target  transition  in  time  period  t,  the  searcher  is  in  cell  i  and 
the  target  is  in  cell  j  and  has  not  been  detected  by  the  first  t 
searches . 

p(0)  €  is  the  initial  joint  searcher-target  distribution 
and  is  assumed  to  be  known  by  the  searcher. 

Hote  that  £jj  Pjj(t)  is  the  probability  that  the  target  ha3  not  been 

detected  by  the  searches  conducted  in  time  periods  I  through  t, 
Also,  Pjj(t)  can  be  calculated  recursively  from  pjjC t - 1 )  by 

conditioning  on  the  searcher  cell  and  the  target  cell  after  the 
search  and  target  transition  in  time  period  (t-1).  Specifically, 
Pjj(t)  -  ( 1  -qj  <5,j)  Pki ( t  - 1 )  SjfjC  t )  j  ,  (1) 

where  <5jj  is  1  if  i=j  and  0  if  i*j. 


3.  The  Search  Problem  as  a  Nonlinear  Program  (NLP) 
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where  Pjj(T)  is  calculated  recurs iue I y  fro#  (1).  The  decision 
variables  are  the  NxN  matrices  S(t) , . . . ,S(T) ,  To  solve  this 
problem,  p(0),  (qi,  ...,qN),  and  T  #ust  be  specified. 

The  four  propositions  which  folio#  establish  certain  properties 
of  the  aboue  nonlinear  progra#.  It  is  noted  that  this  problem, 
while  haying  a  relatively  complicated  objective  function,  has 
constraints  which  are  linear  and  highly  structured. 

Proposition  1:  The  minimus  of  nonlinear  program  ( 2 ) - ( 5 )  is  achieved 
by  a  determini 3t ic  search  plan. 

Proof:  Let  S*  be  an  optimal  search  plan.  (Such  a  plan  exists 

since  the  objective  function  (2)  is  continuous  in  S  and  the 
set  of  feasible  S  defined  by  (3)-(5)  is  compact.)  S*  defines  a 

T-time  period,  M-ce 1 1  Harkov  process  defining  probabi I ist icly 
an  optimal  search  path.  Let  {£i,  ...,£2}  be  the  finite  set  of 
possible  deterministic  search  paths  generated  by  $*.  Each  £k 
has  an  associated  ak  €  where  the  jth  component  of  ak  is  the 
probability  of  nondetection  given  the  searcher  follows  £k  and 
the  target  starts  in  cell  j.  Also  let 
T|j  »  Pij(O) 

be  the  probability  that  the  target  starts  in  cell  j.  Then  the 
probability  of  nondetection  given  £k  is  followed  and  an  initial 
target  distribution  of  T)€lR’xN  is  the  dot  product  riak.  Now  if 
P(£k)  is  the  probability  of  the  searcher  following  path  £k  when 
S*  is  used,  we  have 
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Prob{nondetect ion 


2 

|  S*,^  >  =  ]^p(£k>  r]ak 

i  X]Q*, 

where 

a*  =  arg»inak  (r|ak). 

So  the  deterministic  search  path  associated  with  a*  (and  the 
deterministic  search  plan  which  generated  it)  is  also  opt imal . 

Proposition  2:  S  is  a  deterministic  search  plan  if  and  only  if  S  is 
an  extreme  point  of  the  linear  constraints  (3)-(5), 

Proof:  If  S  i 3  deterministic,  then  by  definition  all  component 
matrices  of  S  must  be  composed  entirely  of  zeros  and  ones. 

Such  an  S  can  not  be  written  as  a  strict  convex  combination  of 
two  other  matrix  series  satisfying  ( 3 ) - ( 5 ) .  So  S  is  an 
extreme  point,  find  if  S  is  not  deterministic,  then  some 
component  matrix  must  contain  a  row  vector  with  two  or  more 
components  strictly  between  0  and  1.  This  row  vector  can  then 
be  expressed  as  a  strict  convex  combination  of  two  other 
distinct  and  feasible  row  vectors.  Thus  S  is  not  an  extreme 
po i nt . 

Proposition  3:  The  objective  function  (2)  is  linear  in  S  when 
constrained  to  any  edge  of  the  simplex  formed  by  ( 3 ) - ( 5 ) . 

Proof:  LetS'  and  $"  be  any  two  adjacent  extreme  point  solutions 
of  the  constraints  (3)-(5).  Specifically,  S'  and  S"  are 
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I 

i 

I 

identical  stochastic  (0,1)  matrix  series  except  in  one  row  of 
one  matrix.  Let  t  be  the  time  period  where  S‘  and  S"  differ, 
and  for  any  Ae[0, 1  ]  let 

Sk  =  AS'  ♦  (1-A)S" 

=  S(1 S(t-1 ),  AS'(t  )♦(  1-A)S”(t ),  S(t  +  1 ),.  . .  ,S(T) 

•  Conditioning  on  the  searcher's  cell  and  the  target’s  cell 
;  before  the  search  in  time  period  t,  we  can  write  the  objective 

function  (2)  evaluated  at  as 

Su  [(^'Qk^ki)  SXik(t)7ji]  Qki(t  +  D,  (b) 

where  is  as  in  (1),  and  Qk|(t  +  1)  is  the  probability  that  the 

•  target  is  undetected  by  searches  in  time  periods  t+l,.,.,T 

given  the  searcher  is  in  cell  k  and  target  is  in  cell  L  at  the 

end  of  time  period  t.  Writing 

i  Sxik(t)  -  AS'ik(t)  ♦  (1-A)S"ik(t),. 

;  and  observing  that  Pjj(t-I)  and  Oki( t + 1 )  are  not  functions  of  A, 

shows  (6)  to  be  linear  in  A. 
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Proposition  4:  fill  basic  feasible  solutions  to  the  nonlinear 
program  (2)-(5)  are  nondegenerate. 


1 


Proof:  The  constraint  matrix  represented  by  (3)  and  (1)  consists 
of  NT  linearly  independent  rows.  So  basic  solutions  will  always 
have  NT  basic  variables,  lie  argue  that  these  basic  variables 
will  always  be  positive.  Each  constraint  consists  of  nonnegative 
variables  summing  to  unity.  So  any  feasible  solution  must 
involve  at  least  one  variable  from  each  constraint.  Since  there 
are  NT  constraints  and  NT  basic  variables,  any  basic  solution 
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must  involve  a  single  basic  variable  in  each  constraint.  Rnd 
furthermore,  since  the  constraints  sum  to  unity,  the  value  of 
each  basic  variable  must  be  I. 


4.  Applying  the  Conuex  Simplex  Method  (CSM) 

The  CSfl,  ihich  is  a  generalization  of  the  simplex  method,  is 
applied  to  nonlinear  programs  with  linear  constraints,  If  the 
current  feasible  solution  is  an  extreme  point  of  the  constraints, 
♦hen  the  CSfl  determines  the  rate  of  change  of  the  objective  function 
along  the  edges  radiating  from  that  extreme  point.  The  current 
solution  then  moves  along  an  edge  with  the  greatest  initial  rate  of 
improvement.  This  movement  continues  until  a  local  optimum  is  found 
along  that  edge  or  an  adjacent  extreme  point  is  reached  (whichever 
occurs  first),  If  there  is  no  edge  along  which  the  objective 
function  can  be  improved,  then  the  current  extreme  point  is  a  ICuhn- 
Tucker  point,  and  a  local  optimum  has  been  found.  The  importance  of 
Propoposit ion  1  is  that  it  guarantees  that  an  extreme  point  which 
does  not  satisfy  the  Kuhn-Tucker  conditions  can  always  be  improved 
with  a  single  iteration  of  the  CSfl,  • 

The  CSh  is  applicable  to  the  nonlinear  program  (2)-(5)  because 
the  constraints  are  linear.  Furthermore,  its  implementation  is 
especially  easy  since  from  Proposition  3  the  objective  function  is 
linear  along  edges  of  the  simplex.  Thus  once  movement  along  an  edge 
is  determined  to  reduce  the  objective  function,  this  movement 
continues  until  an  adjacent  extreme  point  is  reached.  That  is,  no 
one-dimensional  searches  for  local  minima  along  extreme  edges  are 
requ i red , 
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The  CSfl  can  thus  be  implemented  for  this  problem  by  starting  at 
any  feasible  extreme  point,  eualuating  the  object iue  function  at  all 
adjacent  extreme  points,  and  moving  directly  to  the  adjacent  extreme 
point  with  the  smallest  objective  function  value. 

The  calculations  required  to  evaluate  the  objective  function 
(2)  at  any  extreme  point  of  the  constraints  are  simplified  when  the 
searcher's  starting  cell  is  known  with  certainty  (the  usual  case  for 
most  search  applications).  This  occurs  because  the  searcher's 
starting  cell,  together  with  the  determinist ic  search  plan 
associated  with  an  extreme  point,  uniquely  determine  a  search  oath 
s  =  (s( 1 ),..., s(T )) ,  where  s(t)  is  the  searcher's  cell  in  time 
period  t.  find  the  probability  of  nondetection  given  s  is 

T 

PtiD(s)  =  n  (FI  rrfi|)1  (?) 

1=1 

where  T]  is  the  initial  target  distribution,  1  is  a  column  vector  of 
ones,  and  Fs«)  is  the  target  transition  matrix  T  with  row  s(t) 
mu  1 1  i  p  I  i  ed  by  ( I  -qs(t))  ■ 

It  is  also  observed  that,  although  a  general  search  plan  S  is 
an  element  of  IRTxN  ,  a  deterministic  search  plan  can  be  represented 
by  a  TxN  integer  matrix  5,  Each  element  5(t,j)  is  the  searcher's 
cell  in  time  period  t  given  the  searcher's  cell  in  time  period  t-1 
was  j.  S  represents  a  feasible  search  plan  if  and  only  if  each 
S( t ,  i )  is  an  el ement  of  C .. 


The  CSfl,  specifically  tailored  for  the  search  problem  in  (2)- 
(5),  is  the  fol lowing: 

1.  Specify  s(1)  and  an  initial  feasible  3. 

2.  Determine  s  =  (s(1 ) , . . . ,s(T))  from  s(1)  and  3. 

3.  Calculate  PHD(s)  from  (?)  and  set  PHO^,,  =  PHD(s) . 

4.  For  t=1  to  T, 

a.  Uary  S(t,s(t))  over  all  Cs(t)  to  generate 
candidate  adjacent  extreme  points  S'. 

b.  For  each  3‘,  calculate  the  new  search  path  s' 
and  PHD(s' ) , 

c.  If  PHD(s')  <  PHDmin/  save  S'  and  s’,  and  set 
PHDmjn=  PND(s'). 

d.  Cont inue  to  next  t . 

5.  If  no  improvement  i3  achieved  in  PNO^  after  t=T,  then 
STOP.  Otherwise  define  5  as  3‘,  s  as  s’,  and  return 
to  3tep  4. 

Note  that  each  main  iteration  of  this  algorithm  (i.e.,step  4.) 
requires  the  calculation  of  PHD  for  Ij  iCjl  search  paths,  where  |Cj! 
is  the  number  of  cells  in  Cj. 
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5.  The  Storting  Solution 


In  this  sect i on ,  a  procedure  for  generating  an  initial  feasible 
S  is  described,  fl  reasonable  S  would  be  one  that  corresponds  to  a 
myopic  search  path,  fl  myopic  search  path  is  a  path  where  the 
searcher  moves  from  the  cell  searched  in  time  period  t  to  that 
accessible  cell  j  with  the  largest  qjT)j(t),  where 
T)j(t)  =  2j  Pjj(t) 

=  Prob{target  in  cell  j  and  has  not  been 
detected  by  searches  in  time  periods  1,...,t}. 

If  two  or  more  accessible  cells  have  identical  maximal  values 
of  ^ j  T|  j( t ) j  then  the  myopic  policy,  as  used  here,  attempts  to  break 

the  tie  by  selecting  a  cell  with  the  minimum  Euclidean  distance  (or 
some  other  reasonable  norm)  to  a  cell  with  the  maximum  value  of 
qjT"|j(t).  If  a  tie  still  exists,  it  is  broken  randomly.  Uhen  the 

target  is  distant,  this  tie  breaking  procedure  attempts  to  generally 
move  the  searcher  towards  a  favorable  cell  and,  hopefully,  into 
position  for  a  future  detection. 

fl  procedure  is  still  needed  to  complete  the  initial  feasible  S.  0n« 
procedure  would  be  to  assume  that  the  target  distribution  is  a  stationary 
distribution  generated  by  the  target  transition  matrix  f\  (That  is,  the 
target  distribution  is  T)€lR1><N  where  T)  =  T)!”,  r),^0.)  Then  for  each  time 
period  (i.e.,  each  row  of  S)  the  myopic  policy  is  used  to  determine  the 
next  cell  to  be  searched.  This  results  in  an  S  which,  except  for  one 
element  in  each  row,  has  identical  rows.  The  element  which  is  different 


corresponds  to  the  initial  myopic  search  path.  This  starting  solution  was 
tested  and,  for  the  problems  examined,  appeared  to  uiork  well. 


6.  Three  Euampfes 
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Figure  1.  25  cell  search  grid. 


The  target  and  searcher  move  among  the  25  cells  of  Figure  1. 

In  the  next  time  period,  the  searcher  has  access  to  the  cell  just 
searched  plus  all  adjacent  cells.  Cells  are  adjacent  if  they  share 
a  common  side.  Thus,  for  example,  *  {1,2,6}  and 
C3  =  {2, 3, 4, 6}. 

In  each  target  transition,  the  target  remains  in  the  previously 
occupied  cell  j  with  probability  .4  and  moves  to  an  adjacent  cell 
with  probability  .6/mj,  where  mj  is  the  number  of  cells  adjacent  to 

ce  II  j . 

In  the  first  two  examples,  the  searcher  starts  in  cell  I  and 
the  target  in  cell  13.  Assume  that  detection  is  certain  if  the 
target's  cell  is  searched.  That  is,  qk  =  1,  k  =  1,2,..., 25. 
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fl  1 0-t i ne  period  problem  mas  soloed  using  FORTRfiN  77  on  an  I  Bn 
3033  mainframe  computer.  The  following  results  were  obtained: 


Prob.  of  CPU  t ime 

Search  path  Nondetection  (sec) 

flyopic  6  ?  12  13  M  13  18  19  14  9  .4977  .1 

CSM  6  7  8  13  M  19  18  1?  12  7  .4886  1 .42 


In  addition,  a  total  enumeration  routine  was  written  to  find 

the  optimal  solutions.  This  procedure  required  25  CPU  minutes  to 

examine  the  1,225,623  possible  10-time  period  search  paths,  and 

showed  the  CSN  solution  to  be  one  of  12  distinct  optimal  solutions. 

In  this  example  the  myopic  policy  was  close  to  optimal.  Using 

the  CSN  resulted  in  only  a  slight  improvement,  while  requiring 

considerably  more  CPU  time.  The  next  examples  show  these 

observations  are  not  true  in  general. 

In  the  second  example,  a  "fast"  target  was  considered. 

Specifically,  the  probability  that  the  target  in  cell  j  remained  in 

j  was  0,  and  the  probability  that  a  transition  occured  to  any  of  the 

mj  adjacent  cells  was  1/mj.  fill  other  problem  parameters  remained 

the  same.  This  problem  gaive  the  following  results: 

Prob .  of  CPU  t i me 

Search  path  Nondetection  (sec) 

Myopic  6  7  12  12  7  8  9  14  19  18  .4540  .1 

CSM  67  7  8  91419181712  .3868  1.73 

Again,  total  enumeration  showed  the  CSfl  solution  to  be  optimal. 

The  third  example  examined  the  possible  consequences  of  a 
moving  mean  target  position.  Here  the  target  started  in  cell  21  and 
moved  up  or  right  one  cell,  each  with  probability  ,45.  The  target 


remained  in  its  current  cell  with  probability  . J.  This  movement  up 
and  right  continued  until  either  the  top  or  right  boundary  mas 
reached.  Then  with  probability  .9,  the  target  moved  one  cell  up  (if 
currently  in  cell  25,  20  15,  or  10)  or  one  cell  to  the  right  (if  in 
cell  1,  2,  3,  or  1).  Again,  the  target  remained  in  the  current  cell 
with  probability  .1.  Uhen  the  target  reached  cell  5,  it  remained 
there  forever  and  mas  assumed  to  have  escaped.  Except  in  cell  5, 
detection  mas  certain  if  the  target's  cell  mas  searched.  In  cell  5 
the  target  could  not  be  detected.  For  the  10-time  period  problem, 
the  following  results  mere  obtained: 


Prob.  of 

CPU  t i me 

Search  path 

Hondetect i on 

(sec) 

Myopic  6  11 

11  12  13  13  M 

15  15  10 

.2517 

.1 

CSM  6  11 

11  12  13  H  15 

15  10  10 

.1531 

1 .09 

Total  enumeration  ogain  showed 

the  CStl  path  to  be  optimal. 

In  this 

final  example,  the  stationary  target  distribution  has  the  target  in 
cell  5  (the  trapping  state)  with  certainty.  Since  q3  is  0,  the 

modified  myopic  procedure  given  in  the  previous  section  fails  to 
give  a  unique  starting  S.  To  find  a  starting  solution  for  this 
example,  it  mas  assumed  that  q3  mas  an  arbitrarily  small,  positive 
number . 

Finally  it  is  noted  that,  like  most  nonlinear  programming 
solution  procedures,  using  a  poor  starting  solution  can  result  in  a 
poor  final  solution.  In  the  last  example,  setting  S,j  =  j  (except 

for  the  one  element  of  each  row  determined  by  the  myopic  path) 
resulted  in  a  local  optimal  PHD  of  ,2379,  a  slight  improvement  over 
the  myopic  solut ion. 
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